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ABSTRACT 

This brief article reports information gathered at a 
symposium on issues and challenges in assessment and accountability for adult 
English language learners, hosted ny the National Center for ESL Literacy 
Education (NCLE) . Teachers, program administrators, researchers, test 
developers, and policy makers gathered to discuss the field's vision for 
English-as-a-Second-Language (ESL) program accountability and learner 
assessment. The following questions were addressed; What is our vision? What 
do we know? What assessment are available? What do we need? (Adjunct ERIC 
Clearinghouse for ESL Literacy Education) (Author/VWL) 
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Under the Workforce Investment Act (WIA) of 1998, programs that provide 
instruction to adults must be accountable for the outcomes of that instruction 
and must use standardized assessment procedures to document outcomes 
( Workforce Investment Act [WIA] of 1998 , Pub. L. No. 105-220, 112 Stat.936, 
1998). Now, five years later, what progress has the field of adult ESL instruction 
made in accomplishing these goals and meeting these requirements? 



On Friday, May 16, 2003, at a symposium on Issues and Challenges in Assessment 
and Accountability for Adult English Language Learners, hosted by the National 
Center for ESL Literacy Education (NCLE), over 100 teachers, program 
administrators, researchers, test developers, and policy makers gathered to 
discuss the field’s vision for ESL program accountability and learner assessment. 
In addressing the following questions, we were able to review what we have in 
place and assess what we still need. 



• What Is Our Vision? 

• What Do We Know? 

• What Assessments Are Available? 

• What Do We Need? 
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What Is Our Vision? 



Cheryl Keenan, director of the Division of Adult Learning and Literacy in the 
Office of Vocational and Adult Education, U.S. Department of Education, 
described the Department’s vision for assessment and accountability. 

• Any adult who needs or could benefit from education services will have 
access to high quality services in a variety of venues, through a variety of 
service providers, at a variety of times, and in a variety of places. 

• The quality of those services, including the instruction provided, will be 
high and will be based on scientific evidence. 

• Teachers will be well trained and knowledgeable about what they are 
teaching, why they are teaching, and how to teach the learners in their 
classes. 

• Programs, states, and the federal government will be responsible for what 
happens to learners, in terms of educational outcomes and employment, 
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and will be able to track learners’ progress. 

What Do We Know? 

• The learner population in adult ESL programs is large and growing. 

• Programs have varying amounts of experience working with adult English 
language learners. 

• Federal and state policies guide the development and use of assessments. 

• Assessments must be appropriate, valid, and reliable. 

The learner population in adult ESL programs is large and growing. 

The population of adults in the United States who are learning English as a second 
language is large and growing. According to the most recent statistics for program 
year 2000-2001, 42% of all participants (1,119,589 out of a total of 2,673,391) 
enrolled in state-administered adult education programs were enrolled in ESL 
classes (U.S. Department of Education, Office of Vocational and Adult Education, 
Division of Adult Education and Literacy, 2002a). A large percentage of this 
population is enrolled in beginning literacy or beginning ESL classes (U.S. 

Department of Education, Office of Vocational and Adult Education, Division of 
Adult Education and Literacy, 2002b). 

As Mark Kutner (Vice President for Education, American Institutes for Research) 
described, we will know more about this population after the results of the 
National Assessment of Adult Literacy (NAAL) are published in 2004. 

The NAAL household survey of functional literacy has the following components: 

• FAN (Fluency Addition to NAAL) 

• ALSA (Adult Literacy Supplemental Assessment) 

• Health Section 

The FAN will assess the ability of adults to decode letters, words, and numbers; 
comprehend a reading passage; and read with fluency (based on number of words 
per minute read). 

The ALSA will assess the basic reading skills of low-literate English speakers and 
Spanish speakers who are not yet proficient enough in English to take the NAAL, 
based on their ability to read information printed on actual products and 
materials used in everyday life. 

The health literacy of adults learning English will also be assessed, based on their 
ability to understand basic health information. English or Spanish versions of 
these assessments will be used, depending on the language skills of the 
individuals involved. 

Read more about the NAAL . (PowerPoint) 
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Programs have varying amounts of experience working with adult English 
language learners. 

Some states have considerable experience with adults learning English as a 
second language and have put in place effective programs. For example, as Bob 
Bickerton (Director of Adult Education, Commonwealth of Massachusetts) 
described, Massachusetts has a large immigrant population and has been running 
ESL programs for years. Massachusetts has developed adult ESL curriculum and 
learner performance standards , professional standards, assessment policies and 
procedures, a data system for documenting continuous improvement, and a 
performance accountability working group. Different curriculum and learner 
performance standards are developed for different subgroups of learners (e.g., 
English language speakers with limited literacy, English language learners, and 
homeless populations), and resources are devoted to teacher and test 
administrator training. 

In many states (e.g., Arkansas, Nebraska, and Tennessee), the population of 
English language learners is new and has not been served before. Programs in 
these states need research -based information and help from states with more 
experience to guide their development of curriculum frameworks, instructional 
practice, and assessment systems. 

For more information about the population of adults learning English and the 
programs that serve them, see Adult English language instruction in the 21st 
century . 

Federal and state policies guide the development and use of assessments. 

As Carol Van Duzer (NCLE) described, federal policies related to assessment of 
adult learners are in place, and state policies and procedures are being put in 
place. The federal government has developed guidelines for states to follow to 
ensure that instruction is of sufficient quality and duration to help learners meet 
their goals and that tools and procedures for measuring learner progress are 
standardized. The National Reporting System (NRS) (U.S. Department of 
Education, Office of Vocational and Adult Education, Division of Adult Education 
and Literacy, 2001 ) was established to ensure that learner outcomes are reported 
systematically. Within these guidelines, states are selecting assessments that 
programs can use, establishing timeframes for assessments to be given (either at 
specific times during the year or after a given number of hours of instruction), 
and training program staff to administer assessments. 

Assessments need to be aligned with learner goals; with content standards, which 
define what learners need to know and be able to do; and with curriculum and 
instruction. They need to be valid and reliable, with standardized and consistent 
procedures for administration and scoring. 

Program staff need to know the federal and state policies regarding learner 
assessment; they also need to participate in decisions that affect the learners in 
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their programs (such as which assessments should be used, with whom, for what 
purposes, over what time periods, and with what expected outcomes). Program 
administrators need to make sure that staff are trained to administer the 
assessments selected. Teachers need to know what is being assessed and why. 
Test administrators need to follow the guidelines provided with the tests they are 
using. 

Assessments must be appropriate, valid, and reliable. 

Many adult ESL programs use a variety of assessment tools to place learners in 
classes, inform instruction, evaluate learner progress, and report outcomes of 
instruction. These assessments include standardized tests, materials- based and 
teacher-made tests, portfolios, projects, and demonstrations. 

In selecting appropriate tests, we need to know whether we want to assess 
learners’ proficiency or achievement. Achievement testing measures what 
students have learned (what they know and can do) as a result of instruction. Did 
they learn what was taught? Proficiency testing focuses on what students can do 
in real-life situations, whether or not it was learned in the classroom (Buck, 
Byrnes, & Thompson, 1989). The focus in adult ESL education is primarily 
proficiency. 

We know a great deal now about how to assess language learners in valid, 
reliable, and appropriate ways. As Dorry Kenyon (director, Language Testing 
Division, Center for Applied Linguistics) described, the National Research Council 
(with funding from the Office of Vocational and Adult Education) has published a 
b ook on performance as se ssment in adult education , which addresses the 
following issues relevant to those that we face in adult ESL education (National 
Research Council, 2002). 

Appropriate assessments 

In adult ESL education we need to be able to determine the relationship between 
learner outcomes and the various factors that influence those outcomes, which 
include curriculum, classroom instruction, and factors outside the educational 
setting (e.g., learner personality and learning styles, prior education and life 
experiences, and opportunities to use English outside the program). The NRS 
requires that programs report learner outcomes in terms of "educational 
functioning levels,” using valid and reliable assessments. This indicates a need for 
performance assessments which require test takers to demonstrate their skills 
and knowledge in a manner that closely resembles a real life situation or setting. 
Examples of performance assessments include oral or written reports, projects, 
and demonstrations. Peformance assessments are not easy to develop, 
administer, score, and validate. For each test developed, we need to know the 
following: 



• Do the test items elicit what learners know and can do? 

• Does the test administrator know how to give and score the test? 
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• Does the interpretation of scores for the NRS reflect learner knowledge and 
skills in real-life situations? 

Valid and reliable assessments 

Validity is the degree to which the information gained from an assessment 
matches the inferences or decisions that programs make about learners, or 
actions that they take as a result of that information. Whether or not an 
assessment is valid depends on the uses of the outcomes achieved with it. In 
adult ESL education, we need to determine what uses we will make of assessment 
outcomes. Will they be used only to produce reports for the NRS, or will they also 
be used to diagnose learner needs, track learner achievement, and improve 
curriculum and instruction? 

Reliability is the consistency of the measurement when the testing procedure is 
repeated on a different population of individuals or groups. Reliability depends 
first on test developers and distributors, who determine that an assessment is 
reliable. It also depends on those who create the conditions for testing and 
administer and score the test. We need to ensure that the assessments selected 
by states and local programs are used for the purposes, with the populations, and 
in the ways that they were designed to be used (American Educational Research 
Association, American Psychological Association, & National Council on 
Measurement in Education, 1999). 

Read more about standards for assessments . (PowerPoint) 



What Assessments Are Available? 

Programs for adult English language learners an assessment framework and 
standardized assessments of reading, writing, and oral English proficiency 

• EFF Assessment Framework 

• CASAS Assessment System 

• REEP Writing Assessment 

• BEST Plus Oral Language Assessment 

EFF Assessment Framework 

One tool that we have is not an assessment per se, but rather an assessment 
framework. As Regie Stites (program manager, SRI International) described, the 
Equipped for the Future (EFF) assessment framework is aligned with content 
standards for adult education, which define what is important for adults to know 
and be able to do. These standards were developed with broad input from 
learners and practitioners in the field. The assessment framework includes 
standardized tests for accountability purposes and a variety of other assessments 
that can be used for informing and improving instruction. A prototype assessment 
package related to the "Read With Understanding” standard is in development. It 
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will include model standardized tests, technical quality data on the assessments, 
validity and reliability data, test administrator and scorer training materials, 
scoring rubrics, scenarios describing how the assessments might be used, and 
samples of student performance at each of the six EFF performance levels. 
Practitioners will be able to use the assessments in the package or use them as 
models to build their own assessments. 

CASAS Assessment System 

As Linda Taylor (director of assessment development, CASAS) described, the 
CASAS assessment system is designed to integrate curriculum, assessment, and 
instruction into one comprehensive system. It includes standardized multiple- 
choice tests of reading, listening, and math and standardized performance-based 
measures of writing and speaking. These assessments test learner skills from 
beginning literacy to advanced ESL levels. The tests are used in ESL, EL civics, 
citizenship, vocational ESL, employability, and workplace programs. The CASAS 
skill levels are aligned with the NRS proficiency descriptors, and training for test 
administrators is required. 

Read more about the CASAS assessment system . (PowerPoint) 

REEP Writing Assessment 

Suzanne Grant (director of the Arlington Education Employment Program, REEP, 
in Arlington, VA) described a performance-based writing test developed by REEP 
staff that measures writing gains for adult English learners on six levels and is 
aligned with NRS (Beginning to Advanced ESL) and SPL levels (SPL 2 to 8). 
Performance levels are assessed on five dimensions: content, vocabulary, 
organization, structure, mechanics, and voice. Writing tasks (or prompts) for pre- 
and post-testing, warm up activities, and a scoring rubric are included. Research 
carried out by the What Works Literacy Partnership concluded that the 
assessment is valid and reliable and that learner gains can be measured reliably 
after a 12-week period of 100 hours of instruction. The assessment is now being 
tested in intensive and non-intensive programs, closed entry and open entry 
programs, and in individual and group administrations. REEP teachers were 
involved in developing the writing prompts, the test administration procedures, 
scoring rubric, and training materials. This involvement, and use of the 
assessment, has had a positive impact on instruction, because both teachers and 
learners are informed about what the learners are able to do in writing. 

Read More about the REEP writing assessment . (PowerPoint) 

BEST Plus Oral Language Assessment 

BEST Plus is a performance-based assessment, an oral interview to assess the 
functional oral language skills of adults learning English, in the context of 
interpersonal communication. As Dorry Kenyon (Center for Applied Linguistics) 
described, BEST Plus assesses learners from Beginning ESL Literacy to High 
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Advanced NRS levels (SPLs 0-10) in the areas of listening comprehension, 
language complexity, and communication. It was developed by the Center for 
Applied Linguistics, following rigorous test development procedures and with full 
involvement of stakeholders in adult ESL education and assessment. It can be 
used for learner placement, to document and provide a diagnostic report about 
learner progress, for NRS reporting, and for program evaluation. Both computer- 
adaptive and print-based versions of the assessment are available, and training 
for test administrators is required. The score report includes a BEST Plus scale 
score, which is correlated with the appropriate NRS and SPL levels, as well as 
diagnostic information about the learner’s performance. 

Read more about BEST Plus . (PowerPoint) 

What Do We Need? 

We have some very good assessments to choose from. However, these are not 
enough. We need more 

• assessments 

• resources 

• professional development 

• time 

Assessments: We still need assessments that will measure English literacy. There 
are not rigorously researched assessments that demonstrate English language 
learners’ reading and writing skills. 

Resources: In addition to the resources needed to plan and implement effective 
programs, financial resources are needed so that programs and states can 
purchase assessments that are appropriate for program goals and learner goals 
and needs and so that more assessments can be developed. 

Professional development: In order for the assessments to be used reliably, 
professional development is needed so that teachers, testing specialists, and test 
administrators will understand the assessments they use and the reasons they use 
them and will administer the assessments appropriately. 

Time: Programs need time for assessments to be given and for assessment 
procedures and instruments to be aligned with instruction. 

Why is Assessment Important? 

As Cheryl Keenan pointed out, "If we look at the many different subpopulations 
in adult education and at the subpopulations within the ESL portion of adult 
education, we see that not all adult learners are in programs for the same 
reasons, or for the same outcomes, or at the same places along the way. It is our 
responsibility to get them to the next step, whatever that next step is for them.” 
Valid and reliable assessments that adult ESL professionals understand and can 
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use effectively in their programs are a critical part of this responsibility. 
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